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ABSTRACT 


DecisionNet  is  an  online  Internet-based  repository  of  decision 
technologies.  It  links  remote  users  with  these  technologies  and  provides  a 
directory  service  to  enable  search  and  selection  of  suitable  technologies.  The 
ability  to  retrieve  relevant  objects  through  search  mechanisms  is  basic  to  any 
repository’s  success  and  usability  and  depends  on  effective  classification  of  the 
decision  technologies.  This  thesis  develops  classification  methods  to  enable 
indexing  of  the  DecisionNet  repository. 

Existing  taxonomies  for  software  and  other  online  repositories  are  examined. 
Criteria  and  principles  for  a  good  taxonomy  are  established  and  systematically 
applied  to  develop  DecisionNet  taxonomies.  A  database  design  is  developed  to 
store  the  taxonomies  and  to  classify  the  technologies  in  the  repository.  User 
interface  issues  for  navigation  of  a  hierarchical  classification  system  are  discussed. 
A  user  interface  for  remote  World  Wide  Web  users  is  developed.  This  user 
interface  is  designed  for  browsing  the  taxonomy  structure  and  creating  search 
parameters  online.  Recommendations  for  the  implementation  of  a  repository 
search  mechanism  are  given. 
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I.  INTRODUCTION 


The  widespread  popularity  and  growth  of  the  Internet  and  the  World  Wide  Web 
has  created  new  opportunities  for  the  sharing  of  data  and  executable  software  between 
users  worldwide.  One  such  application  is  DecisionNet,  an  online  repository  of  decision 
support  systems.  The  general  area  of  research  this  thesis  addresses  is  the  implementation 
issues  involved  with  establishing  an  online  repository.  The  repository  is  distributed 
(objects  are  stored  remotely  on  other  servers  worldwide)  but  cataloged  at  the  Naval 
Postgraduate  School  and  accessed  centrally  by  users  worldwide  through  DecisionNet. 

The  DecisionNet  system  consists  of  the  decision  technologies,  a  database  of  information 
describing  those  objects  and  software  agents  that  handle  consumer  and  provider 
administrative  information  and  perform  browsing  and  search  mechanisms.  (Bhargava, 

King  and  McQuay,  1995) 

The  objective  of  this  research  is  to  develop  and  implement  taxonomies  for 
classifying  the  technologies  for  the  purpose  of  indexing  and  retrieval  by  providers  and 
users  of  the  system.  An  effective  classification  scheme  will  allow  users  to  search  and  get 
results  scored  by  relevancy,  and  will  be  useful  in  doing  automatic  registration  of  objects  by 
remote  providers. 

A.  DECISION  SUPPORT  SYSTEMS 

Decision  Support  Systems  (DSS’s)  are  used  by  managers  and  planners  to  perform 
modeling,  simulation  and  expert  analysis  of  variables  and  conditions  in  support  of  decision 
making.  DSS’s  are  software  programs  that  employ  computational  methods  such  as 
mathematical  algorithms  and  artificial  intelligence  techniques  to  determine  the  best  action 
to  take  using  available  information.  DSS’s  are  valuable  for  managers  and  decision  makers 
in  fields  ranging  from  evaluating  investment  opportunities  for  personal  use  to  wargaming 
and  logistics  modeling  for  the  Department  of  Defense  (DoD).  A  number  of  decision 
support  systems  apply  results  from  research  in  Operations  Research  (OR)/Management 
Sciences  (MS)  using  techniques  such  as  optimization  and  modeling. 

Some  examples  of  optimized  results  given  resources  under  constraint  are: 
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•  Scheduling  and  assignment  of  police  officers 

•  Reducing  transportation  time  and  fuel  cost 

•  Reducing  fuel  cost  to  DoD  for  transport  of  supply/personnel 

•  Inventory  management 

•  Determining  risk  tolerance  and  investment  selection 

•  Production  planning 

•  Selection  of  locations  for  service  providers,  utility  stations,  refueling  stations 

These  examples  are  illustrative  of  civil  government  functions,  military,  commercial 
and  private  uses  of  DSS’s  and  their  use  in  improving  and  analyzing  decisions.  Potential 
users  of  DSS’s  and  DecisionNet  include  university  researchers  and  students,  military  and 
government  planners,  private  industry  staff  and  other  Internet  users. 

B.  DECISIONNET:  WHY  NETWORK  BASED  DECISION  SUPPORT 

SYSTEMS? 

The  Internet  has  become  an  important  medium  for  the  interchange  of  ideas,  data, 
software  and  other  unique  problem  solving  applications.  A  DSS  repository  such  as 
DecisionNet  will  link  users,  operations  research  problem  solvers  and  information  system 
developers. 

DSS’s  tend  to  be  narrowly  focused  on  a  specific  problem  or  industry  (Sprague, 
1980)  and  while  they  are  effective  tools  for  decision  makers  they  may  be  costly  to  develop 
or  purchase.  The  ability  to  share  software  on  a  network  such  as  the  Internet  means 
software  is  made  available  for  use  without  necessarily  requiring  purchase.  Software  is 
essentially  rented  and  intellectual  property  rights  or  solution  methods  remain  in  the  hands 
of  the  developer  (Bhargava,  King  and  McQuay,  1995). 

Using  the  Internet,  data  is  sent  to  the  objects  (models,  algorithms,  and  decision 
support  systems)  or  responded  to  online  through  the  use  of  forms  and  browsable  World 
Wide  Web  pages  in  a  client-server  configuration.  For  some  DSS’s,  data  is  sent  by  the  user 
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to  the  remote  decision  technology  where  the  answer  is  computed  and  sent  back  to  the 
user.  For  other  DSS’s,  there  is  an  interactive  text  based  dialogue  between  the  client  and 
server.  For  example,  the  Water,  Soil,  Hydro-Environmental  Decision  Support  System 
(WATERSHEDSS)  is  a  knowledge  based  water  quality  decision  support  system.  To  use 
the  DSS  the  user  navigates  a  decision  tree  to  arrive  at  a  text  based  answer.  For  both 
types,  the  DecisionNet  system  provides  the  directory  service  and  “mediates  the  initial 
transactions  between  users  and  different  technologies.”  (McQuay,  1995) 

The  Internet  is  based  on  the  Transmission  Control  Protocol/Intemet  Protocol 
(TCP/IP)  as  a  standard  for  transmitting  all  kinds  of  files,  including  executable  software,  e- 
mail,  data  sets  and  World  Wide  Web  pages.  These  tools  and  infrastructure  make  the 
Internet  the  perfect  platform  for  this  type  of  application. 

The  DoD  has  also  adopted  the  TCP/IP  standards  for  the  common  operating 
environment  (COE)  systems  of  the  future.  By  developing  DecisionNet  within  the 
standards  of  these  Command  and  Control  Systems,  a  parallel  system  could  be  developed 
to  function  operationally  on  military  networks,  similar  to  the  Internet.  Such  a  system 
might  primarily  store  military  DSS’s  for  military  decision  makers.  (Defense  Information 
Systems  Agency,  1994) 

C.  DECISIONNET  AS  A  DIGITAL  LIBRARY 

DecisionNet  is  designed  to  be  a  digital  library  for  DSS  technologies.  In  order  to 
keep  track  of  the  objects  in  the  DecisionNet  repository,  the  meta-information  about 
objects  must  be  organized  logically  to  facilitate  searching  of  the  database  by  users. 

Digital  libraries  share  important  characteristics  with  physical  libraries  in  use  today. 
Libraries  are  repositories  for  many  types  of  objects.  In  order  to  organize  these  objects  for 
quick  and  meaningful  retrieval  (based  on  a  search  by  a  user,)  they  are  described  and 
classified  by  their  characteristics.  The  library’s  collection,  like  DecisionNet’ s  repository,  is 
represented  by  a  database  and  the  user  interacts  with  a  system  that  accesses  the  database. 

Physical  libraries  generally  keep  many  types  of  objects:  newspapers,  magazines, 
films,  videotapes,  books,  pamphlets,  etc.  .  The  characteristics  on  which  objects  are 
classified  include: 
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•  Each  object  has  a  primary  author,  editor  or  creator. 

•  Each  object  has  a  publisher  or  distributor  and  date  of  creation  or  publication. 

•  Each  object  has  a  title. 

•  Each  object  is  classified  by  subject.  (Libraries  use  taxonomies  like  a  subject  tree 

or  the  Dewey  Decimal  System  for  non-fiction  works.) 

These  characteristics  or  properties  together  describe  one  unique  object.  Even  if 
the  only  difference  between  one  object  and  another  is  edition  number  (different  date  of 
publication  with  some  modifications  and  improvements, )  no  two  objects  are  identical. 

As  in  a  physical  library,  the  contents  of  a  digital  library  are  identified  by  their 
unique  characteristics.  The  user  has  to  be  introduced  to  a  subject  tree  or  taxonomy  that 
he  navigates  to  locate  the  object  he  is  seeking. 

We  have  determined  the  unique  characteristics  of  objects  contained  within 
DecisionNet  to  be: 

•  Problem  area-  the  specific  task  performed  by  the  DSS 

e.g.  loan  amortization,  water  quality  determination 

•  Functional  area  -  the  specific  departments  within  an  organization 

e.g.  administration,  operations,  legal,  sales 

•  Industry  type  -  the  type  of  work  a  business  or  organization  performs 

e.g.  engineering,  manufacturing 


•  Organization  Type-  one  of  the  five  types  of  user  orientations  for  which  the 

application  was  designed:  military,  government,  non-profit,  commercial,  or 
individual 


•  Object  type-  one  of  the  four  types  of  objects  DecisionNet  stores:  data  set, 

algorithm,  model,  or  functional  decision  support  system 

•  Solution  Method-  the  mathematical  or  deductive  method  of  the  DSS 

e.g.  optimization,  numerical  math,  spreadsheet  modeling 

Some  users  of  physical  libraries  go  in  with  an  idea  about  at  least  one  of  the 
characteristics  they  are  looking  for  and  some  knowledge  of  the  subject  area  and  browse  (a 
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computer  or  a  shelf  of  the  library)  until  they  find  the  best  one  or  two  objects.  Other  users 
know  exactly  what  they  want  and  only  follow  the  signs  in  the  library  pointing  them  to  its 
location.  First  time  users  need  to  start  with  a  tour  of  all  available  services. 

Similarly,  digital  library  users  will  search  for  topics  in  different  ways.  A  casual 
Internet  user  looking  for  a  simple  DSS  like  a  mortgage  calculator  may  use  the  system  and 
have  no  knowledge  or  interest  about  how  such  a  task  is  performed.  Another  user,  perhaps 
an  undergraduate  student  taking  a  computer  course,  may  need  to  find  out  about  decision 
support  systems  but  not  have  any  real  idea  about  the  solution  method  they  seek.  Some 
users  will  just  want  to  browse  the  repository  to  see  what  is  available.  Operations 
research  and  computer  science  researchers  may  seek  a  specific  computational  solution 
method  regardless  of  the  application  it  performs. 

This  thesis  explores  the  problem  of  how  to  organize  and  classify  the  objects  in 
DecisionNet  repository  based  on  their  unique  characteristics:  problem  area,  functional 
area,  industry,  organization  type,  object  type,  and  solution  method.  Effective 
classification  of  the  elements  of  the  repository  will  allow  users  to  browse  the  repository 
for  what  they  are  looking  for  by  themselves  and  allow  for  searches  based  on  the 
characteristics  of  the  object  they  seek. 

D.  PREVIOUS  WORK  DONE  ON  DECISIONNET 

Prior  to  this  thesis,  work  on  DecisionNet  identified  the  required  primary 
capabilities  and  information  flow.  The  first  prototype  automated  many  of  the  registration 
functions  and  built  a  search  mechanism  for  registered  DSS  technologies  by  keyword 
search  of  flat  files  containing  all  information  available  about  the  objects.  Both  CAPT  Dan 
McQuay  (McQuay,  1995)  and  LT  Andy  King  (King,  1995),  who  developed  the  first 
prototype,  highlighted  the  need  for  a  taxonometric  classification  scheme  to  implement 
relevancy  searches. 

E.  DEFINITION  OF  KEY  TERMS  AND  CONCEPTS 

1.  DecisionNet  System  Components 

The  DecisionNet  system  is  comprised  of  three  basic  parts:  informational  HTML 
pages  and  software  agents,  a  database,  and  the  repository.  The  repository  is  the 
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collection  of  decision  technologies  that  are  registered  in  DecisionNet.  The  software 
agents  handle  the  user’s  input  through  interaction  with  the  database  tables.  The  database 
contains  information  about  the  decision  technologies  that  the  search  mechanisms  query  to 
match  the  user’s  request.  The  system  directs  the  user  to  the  objects  in  the  repository  that 
match  the  search  request.  Users  of  the  system  are  classified  as  consumers  or  providers. 
Consumers  search  the  database  looking  for  a  decision  technology  to  use  and  providers 
register  technologies  others  might  want  to  use. 

2.  Taxonomy  Definitions 

A  taxonomy  is  an  abstract  division  of  objects  into  ordered  groups  or  categories 
based  on  their  characteristics.  One  of  the  most  familiar  taxonomies  is  used  in  biology  to 
divide  all  organisms  into  one  of  two  kingdoms:  animal  and  plant.  Within  a  kingdom, 
objects  are  broken  down  into  phyla,  then  classes,  orders,  etc.  as  shown  in  Fig.  1.1. 


Kingdom 

.  _ 

Phylum 

1 

Class 

1 

Order 

1 

Family 

Genus 

Species 

1 

Figure  1 . 1  Taxonomy  Applied 
to  Organisms 


Each  level  is  considered  a  “taxon”,  a  taxonomic  category  or  group.  To  list  family 
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types  and  genus  types  together  would  be  to  confuse  levels  of  specialization.  Taxonomies 
are  similar  to  family  trees  because  properties  of  inheritance  exist.  Organisms  within  the 
same  order  have  similar  characteristics  in  common  but  belong  to  different  geni  or  species 
because  of  their  distinct  characteristics. 

3.  Object  Types  Handled  by  DecisionNet 

The  DecisionNet  repository  catalogs  four  types  of  objects:  data  sets,  algorithms, 
models  and  functional  decision  support  systems. 

F.  SEARCH  TYPES 

In  assessing  how  searches  of  the  database  should  be  processed  there  are  two 
primary  considerations,  how  the  search  is  executed  and  how  the  database  is  indexed.  One 
search  method  is  a  batch  or  real  time  search  of  the  database  that  involves  searching  the 
database  on  each  request.  The  results  are  always  current,  and  this  method  is  appropriate 
for  a  dynamically  changing  database.  Another  search  method  is  to  index  the  database  and 
make  updates  on  a  weekly  or  daily  basis.  This  is  a  faster  approach,  but  not  necessarily 
completely  current  or  perfectly  accurate.  Current  practice  on  the  Internet  is  that 
repositories  that  have  a  database  interface  do  batch  processing  of  requests.  Online 
Internet  search  engines  primarily  do  queries  to  a  central  database  that  is  periodically 
updated.  These  updates  are  made  by  traveling  spiders  or  indexing  robots.  The  appropriate 
search  method  depends  on  the  objective  of  the  administering  organization.  Is  it  to 
minimize  retrieval  time?  Maximize  accuracy  and  thoroughness?  Are  there  serious 
constraints  on  the  storage  space  and  processing  power  of  the  database?  Each  of  these 
leads  to  a  different  architecture  of  the  database. 

Another  consideration  is  that  a  database  can  be  indexed  for  searching  in  two  ways. 
The  first  method  is  keyword  searching  the  entire  database  for  a  certain  string  or  term.  For 
some  keyword  searches  there  will  be  many  exact  matches.  Because  synonyms  or  a  similar 
concept  may  be  used  to  classify  the  object  under  a  keyword  search  orientation  there  is  a 
chance  nothing  would  be  retrieved  and  that  relevant  objects  would  not  be  brought  to  the 
attention  of  the  user.  The  second  way  is  to  compute  relevancy  based  on  classification  of 
objects  under  an  indexing  scheme.  In  order  to  include  all  relevant  items  and  to  allow 
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users  to  find  what  they  are  looking  for,  a  relevancy  calculating  scheme  is  used.  To 
compute  relevancy  a  classification  scheme  for  each  of  the  six  classifying  characteristic 
must  be  developed.  Ideally  every  conceivable  choice  under  each  of  these  characteristics 
would  be  identified  to  build  a  mutually  exclusive  and  collectively  exhaustive  list  of 
selections.  To  take  this  organization  one  step  further,  we  want  to  teach  the  search 
mechanisms  the  taxonomy  of  each  classifying  characteristic  so  that  it  will  retrieve 
something  related  if  the  exact  query  can  not  be  satisfied. 

G.  ORGANIZATION  OF  THESIS 

This  thesis  is  organized  into  five  chapters  and  one  appendix.  The  first  chapter 
contains  the  introduction,  an  overview  of  the  DecisionNet  concept,  a  discussion  of  the 
work  done  on  system  design,  and  a  discussion  of  search  mechanisms  for  a  repository.  The 
second  chapter  examines  the  relationship  between  the  objects  in  the  repository  and  their 
representation  in  the  database.  Additionally,  the  concept  of  taxonomies  is  introduced  as 
the  primary  indexing  scheme,  an  analysis  of  current  practice  is  done  and  the  criteria  for  a 
good  taxonomy  are  developed.  Chapter  III  discusses  the  methodology  used  to  develop 
taxonomies  for  four  of  the  six  characteristics.  Appendix  A  is  the  expanded  view  of  the 
solution  method  taxonomy  that  is  shown  in  Chapter  III.  Chapter  IV  examines  the 
implementation  of  the  taxonomies  using  available  Internet  software  tools.  That  chapter 
also  discusses  principles  of  user  interface  design  considered  in  developing  the  DecisionNet 
system  s  user  interface.  Chapter  V  is  the  thesis  conclusion  and  offers  suggestions  for 
future  work  on  the  DecisionNet  classification  scheme  and  taxonomy  development. 
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H.  TAXONOMIES:  ROLE  &  CRITERIA 


A.  SEARCHING  A  DATABASE  -  HOW  TO  ORGANIZE  A  SITE  FOR 

OUTSIDE  USERS 

The  current  practice  for  Internet  online  pages  is  to  include  text,  graphics  and 
logos,  product  information  and  personnel  and  point  of  contact  information.  Increasingly 
organizations  are  seeing  the  Internet  as  a  forum  for  network  publishing.  For  example, 
government  publications  are  increasingly  available  online  because  this  is  an  inexpensive 
way  to  publish  them.  Network  publishing  provides  access  by  a  greater  number  of  users 
who  were  previously  limited  by  physical  proximity  to  a  database  or  libraries  holdings. 
Examples  include: 


•  Digital  Libraries-  Library  of  Congress  and  university  libraries  worldwide 

•  Bookstores/  video  and  music  stores  that  make  some  of  their  products  available 
for  worldwide  users 

•  Government  Printing  Office-  legislation,  Census  Information,  Government 
Accounting  Office  (GAO)  reports 

These  repositories  have  the  common  need  to  classify  their  collections  for  indexing 
and  retrieval  by  remote  users.  The  Wide  Area  Information  Service  (WAIS)  is  a  software 
package  that  indexes  the  entire  contents  of  a  server  and  has  an  online  search  engine. 
Results  are  scored  on  the  number  of  occurances  of  a  word  (keywords  entered  by  the  user) 
within  the  documents  that  meet  certain  search  parameters  (e.g.  written  after  1993) .  For 
example,  the  Government  Accounting  Office  (GAO)  has  put  all  its  reports  online  and  they 
are  completely  indexed  and  available  for  search  and  retrieval  by  keyword  using  WAIS. 
This  is  effective  because  the  objects  are  homogenous  and  keyword  search  effectively 
distinguishes  the  primary  differences  between  the  objects  in  the  repository.  But  it  does  not 
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solve  the  synonym  problem  mentioned  earlier.  More  importantly,  this  kind  of  indexing 
scheme  is  not  appropriate  for  DecisionNet’s  consideration  of  more  than  one  characteristic 
by  the  search  mechanism. 

B.  USING  TAXONOMIES  TO  CLASSIFY  AND  SEARCH  FOR  OBJECTS 

To  give  a  preview  of  the  DecisionNet  taxonometric  classification  scheme,  there  are 
six  characteristics  every  object  has  and  can  be  described  by  (the  horizontal  axis  in  the 
following  figure).  Under  each  characteristic  is  a  hierarchical  taxonomy  (represented 
vertically).  The  depth  of  the  classification  represents  how  many  levels  of  specialization 
exist  within  that  type.  The  more  classification  and  specialization  assigned  to  an  object  the 
greater  the  capability  for  finer  and  more  accurate  searching. 

Figure  2. 1  illustrates  that  decision  technologies  possess  characteristics  and  that  a 
relationship  exists  between  those  objects  and  the  taxonomy  of  each  characteristic. 
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Figure  2. 1  Relationship  Between  Decision  Technology  Characteristics  and 
Taxonomy  Scheme 
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Essentially,  the  characteristics  exist  as  abstract  classes  whether  or  not  there  are  any 
objects  with  those  characteristics  currently  registered  in  the  repository.  In  Figs.  2. 1  and 
2.2  Decision  Technology  1  has  the  following  attributes:  problem  area  (Ala),  functional 
area  (C2),  industry  (D),  organization  type  (C),  object  type  (D),  and  solution  method  (B) . 
Decision  Technology  2  has  the  following  attributes:  problem  area  (Blc),  functional  area 
(A2),  industry  (D),  organization  type  (E),  object  type  (B)  and  solution  method  (G).  In 
Fig.  2.2  the  user’s  search  parameters  are  scored  against  the  decision  technologies  in  the 
repository  (as  they  are  represented  in  the  database).  The  search  mechanism  computes 
relevancy  by  determining  the  distance  between  nodes  representing  the  users  search 
parameters  and  the  decision  technologies  in  the  repository  . 


Figure  2.2  Searching  DecisionNet  Repository  by  Characteristic 
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Using  Decision  Technology  1  and  2  for  this  example,  a  simple  search  mechanism 
compares  the  user’s  request  against  the  characteristics  of  the  two  objects  and  returns  a 
rating  as  to  which  is  more  relevant.  Relevancy  is  determined  by  computing  the  distance 
between  a  specific  decision  technology  (DT)  and  user’s  search  parameters  (SP)  to  a 
common  parent  in  each  characteristic. 


User  SP 

DT  1 

DT  2 

Distance  User’s 

SP  to  DTI 

Distance  User’s 

SP  to  DT2 

Problem  Area 

B1 

Ala 

Blc 

3 

1 

Functional  Area 

C3 

C2 

A2 

1 

3 

Industry 

C 

D 

D 

1 

0 

Organization  Type 

C 

C 

E 

0 

1 

Object  Type 

D 

D 

B 

0 

1 

Solution  Method 

F 

B 

G 

1 

1 

Total  Distance 

5  nodes 

7  nodes 

Table  2. 1  Distance  Between  Decision  1 

fechnologies  and  User’s  Search  Parameters 

Decision  Technology  1  is  more  relevant  to  the  users  search  than  Decision 
Technology  2  and  would  be  scored  accordingly. 

C.  LITERATURE  REVIEW  -  CURRENT  PRACTICE 

One  of  the  most  important  contributions  of  the  Internet  is  the  easy  access  it 
provides  to  documents,  algorithms,  software  modules,  data  sets  and  executable  programs. 
A  number  of  repositories  have  been  established  by  academic  and  commercial 
organizations  and  these  repositories  face  the  organization  of  information  and  search 
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problems  discussed  previously.  There  is  little  information  published  that  is  directly  related 
to  implementing  a  network  based  repository. 

There  are  many  keyword  search  engines  on  the  Internet  and  very  few  classification 
schemes.  Two  repositories  that  do  effectively  index  their  collections  with  classification 
schemes  are  Yahoo  and  the  Guide  to  Available  Mathematical  Software  (GAMS).  The 
first,  Yahoo,  is  a  “searchable,  browsable,  hierarchical  index  of  the  Internet”  by  subject. 

The  second,  GAMS,  is  an  index  of  algorithms  by  solution  method.  The  primary 
contribution  of  GAMS  is  its  problem  classification  system.  The  GAMS,  developed  by  the 
National  Institute  of  Standards  and  Technology  (NIST),  is  a  73 6  node  “tree  structured 
taxonomy  extending  to  seven  levels.  Each  child  node  in  the  tree  denotes  a  more  specific 
instance  of  the  mathematic  problem  represented  by  its  parent”.  (Boisvert,  1991) 

Many  academic  and  some  commercial  organizations  are  developing  online 
repositories  of  software  or  algorithms  that  are  organized  using  keyword  searches  or 
adopting  the  GAMS  problem  classification  scheme.  The  Numerical  Algorithm  Group 
(NAG)  and  Netlib  (developed  in  conjunction  with  Oak  Ridge  National  Laboratory  and  the 
AT&T  Bell  Labs)  both  use  GAMS  as  the  single  classification  characteristic  for  their 
repositories.  The  DecisionNet  repository,  however,  will  keep  more  than  just  algorithms. 

It  will  also  classify  models  and  DSS’s  that  are  tailored  to  a  specific  application  and  user 
group. 

D.  TAXONOMIES  WITHIN  A  REPOSITORY 

A  taxonomy  is  defined  as  the  division  into  ordered  groups  or  categories. 
Systematics  or  a  taxonomic  organization  of  a  field  is  undertaken  to  “structure  the  body  of 
knowledge  that  constitutes  a  field  once  it  has  reached  a  certain  level  of  maturity”  (Glass 
and  Vessey,  1994).  This  is  done  to  break  a  field  down  for  further  study,  to  understand  the 
elements  of  a  field,  and  to  predict  future  areas  of  study  or  to  create  specialization. 

Taxonomies  are  essentially  hierarchies  where  objects  are  classified  from  general  to 
specific.  Once  we  have  decided  the  dimensions  that  describe  the  objects,  what  will  the 
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taxonomy  of  options  look  like?  A  tree  is  the  most  natural  for  a  classification  scheme 
because  it  “allows  for  refinement  in  mature  and  young  subject  areas”.  Hierarchical  based 
classification  gives  the  users  the  flexibility  to  refine  the  specification  by  “using  the 
classification  as  a  decision  tree”  (Boisvert,  Aug.  1994)  and  allows  a  way  for  matters  of 
relevance  and  relatedness  to  be  determined  so  that  they  can  be  built  into  the  search 
routine.  Classes  will  be  linked  and  may  be  “siblings”  or  “children”  of  other  areas. 
Selections  should  also  be  mutually  exclusive  and  collectively  exhaustive. 

E.  CURRENT  PRACTICE  FOR  DSS  TAXONOMIES 

Consider  current  practice  and  academic  consideration  of  DSS  taxonomies. 

A  literature  review  revealed  that  the  Decision  Support  Systems  community  of  the 
Information  Systems  field  and  the  software  development  side  of  the  OR/MS  field  think  of 
decision  support  systems  in  broad  terms.  Table  2.1  illustrates  one  method  for  classifying 
DSS’s: 


TvDe  of  problem 

Generic  Approach 

Nature  of  Choice 

Strategic  focus 

Analyzing 

Optimal 

Tactical  focus 

Standardizing 

Acceptable 

Operational  focus 

Automatizing 

Rewarding 

Table  2.2  (After  Nagel,  1993)  Decision  Support  System  Problem  Types 


This  view  considers  structured  (operational),  semi-structured  (tactical)  and 
unstructured  (strategic)  decisions  as  an  evaluation  between  degree  of  risk  and  risk 
aversion,  payoff  or  loss  potential  and  certainty  of  information. 

The  Glass  and  Vessey  paper  “Towards  Taxonomies  of  Application 
Domains”(1994)  considers  several  taxonomies  in  use  by  the  DoD,  software  development 
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companies  and  other  software  standards  making  bodies.  Their  analysis  of  these 
taxonomies  is  that  software  was  broken  down  as  either  application  oriented  or 
infrastructure  oriented  and  within  those  domains  it  was  considered  to  be  either  problem 
focused  (industry  or  application  -  called  problem  area  in  this  paper)  or  solution  focused 
(called  solution  method  in  this  paper).  Another  DSS  researcher  categorized  decision 
support  systems  as: 

•  Approaches  based  on  OR/MS 

•  Spreadsheet  based 

•  Expert  system  based  (rule  based/function  based)  (Nagel,  1993) 

While  these  may  be  effective  for  considering  a  single  DSS  or  comparing  one  with 
another  it  is  not  an  effective  way  to  describe  and  search  a  repository  of  DSS’s.  However, 
Nagel’s  generalization  was  a  starting  point  for  creating  a  taxonomy  based  on  solution 
method  as  discussed  later. 

Each  of  these  examples  is  not  suitable  for  classifying  a  decision  support  system  in  a 
way  that  helps  the  user  find  what  they’re  looking  for  in  a  repository  of  many  objects. 

F.  CRITERIA  FOR  TAXONOMIES 

Dr.  Boisvert,  the  creator  of  GAMS  writes  “To  be  effective  a  classification  system 
must  have  the  following  properties: 

•  problem-orientation 

•  variable-level  tree  structure  for  specialization  and  generalization  of  topics 

•  active  maintenance  by  system  administrators  for  refinement”  (1991  ) 

Based  on  these  classifications  and  other  examples  it  was  decided  that  the 
following  criteria  should  be  used:  a  taxonomy  be  principled,  scalable,  maintainable  and 
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robust.  Principled,  means  that  it  consists  of  elements  that  are  distinct  instances  of  the 
same  attribute  at  a  given  level  and  those  distinctions  are  based  on  the  principles  that  apply 
to  that  characteristic.  Maintainable  refers  to  the  capacity  of  each  taxon  to  be  thorough 
and  create  significant  distinction  between  the  categories  while  not  being  stifling  at  the 
highest  level.  A  scalable  taxonomy  is  one  that  although  it  is  thorough  at  each  level  the 
structure  is  built  in  such  a  way  that  further  refinement  or  specialization  can  be  done  at  the 
lower  levels  without  changing  the  original  structure.  Robust  refers  to  the  completeness  of 
the  original  framework  in  describing  the  problems  the  repository  will  be  applied  to  so  that 
the  original  architecture  covers  the  fields  and  data  sorting  and  classifying  needs  of  the 
system  in  its  future. 
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Bd.  DEVELOPMENT  OF  TAXONOMIES  FOR  DECISION  TECHNOLOGIES 


The  primary  objective  of  this  thesis  is  to  identify  the  classifying  characteristics  or 
meta-information  required  for  each  decision  technology  in  the  database  and  then  develop  a 
taxonomy  for  each  characteristic.  Providers  use  the  taxonomy  to  register  objects  and 
consumers  use  them  to  search  the  database. 

The  six  ways  in  which  objects  can  be  classified  or  searched  are: 

•  problem  area 

•  functional  area 

•  industry 

•  organization  type 

•  object  type 

•  solution  method 

The  object  types  to  be  handled  by  DecisionNet  was  settled  on  previously  and  the 
organization  type  was  easily  classified  into  five  types  and  did  not  require  further 
classification,  but  the  other  four  required  taxonomy  development. 

For  the  task  of  creating  a  taxonomy  for  each  characteristic  there  is  a  great  deal  of 
material  on  existing  indexing  methods.  Some  of  the  materials  available  includes  the 
subject  classification  scheme  used  in  the  Operations  Research/  Management  Science  Index 
(Tolle,  1988) ,  the  Encyclopedia  of  Operations  Research  and  Management  Science  (Gass 
and  Harris,  1995)  and  the  Guide  to  Available  Mathematics  Software  (GAMS).  Table  3.1 
shows  the  GAMS  framework  at  the  highest  level  with  amplification  of  the  optimization 
class  to  three  additional  levels. 

To  build  a  taxonomy  for  the  other  characteristics  existing  classification 
frameworks  were  studied.  For  problem  area,  functional  area  and  industry  we  began  with 
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types  of  businesses  and  services  and  their  departmental  components  and  the  specific  tasks 
they  performed.  The  OR/MS  community  has  done  this  in  an  effort  to  classify  the  articles 
written  by  members  of  their  community  and  developed  a  primary  subject  classification 
system  used  in  the  OR/MS  indices.  However  the  subject  areas  are  not  of  the  same  taxon, 
that  is  to  say,  at  the  same  taxonomic  level  or  within  the  same  classification  characteristic. 
However  it  provided  a  valuable  starting  place  for  topics  and  industries  involved  with 
Decision  Support  Systems. 

All  fields  and  disciplines  regardless  of  whether  they  may  or  may  not  have  any 
current  association  with  decision  support  systems  were  considered.  All  problem  areas  or 
functions  that  potentially  could  be  analyzed  or  modeled  with  the  aid  of  a  decision  support 
system  were  included.  This  was  to  keep  the  taxonomies  robust. 

Glass  and  Vessey’s  paper  provided  a  number  of  taxonomies  developed  by  industry 
software  leaders  broken  down  by  application  type  and  their  component  tasks.  The  paper 
argues  for  the  importance  of  “applying  strong  application-dependent  methods  in  the 
computing  field”  particularly  in  expert  systems  and  knowledge  based  systems  (Glass  and 
Vessey,  1994).  Additionally,  the  U.S.  Census  and  U.S.  Patent  Office  list  very  specific 
business  types,  functions  and  methods  of  industries. 


A.  Arithmetic,  error  analysis 

B.  Number  theory 

C.  Elementary  and  special  functions 

D.  Linear  Algebra 

E.  Interpolation 

F.  Solution  of  nonlinear  equations 

G.  Optimization 

Gl.  Unconstrained 

Gla.  Univariate 

Gl al .  Smooth  function 

Gla2.  General  function  (no  smoothness  assumed) 

Gib.  Multivariate 

Glbl .  Smooth  function 
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Glb2. 

General  function  (no  smoothness  assumed) 

G2. 

Constrained 

G2a. 

Linear  programming 

G2al. 

Dense  matrix  of  constraints 

G2a2. 

Sparse  matrix  of  constraints 

G2b. 

Transportation  and  assignments  problem 

G2c. 

Integer  programming 

G2cl. 

Zero/one 

G2c2. 

Covering  and  packing  problems 

G2c3. 

Knapsack  problems 

G2c4. 

Matching  problems 

G2c5. 

Routing,  scheduling,  location  problems 

G2c6. 

Pure  integer  programming 

G2c7. 

Mixed  integer  programming 

G2d. 

Network  (for  network  reliability  search  class  M) 

G2dl. 

Shortest  path 

G2d2. 

Minimum  spanning  tree 

G2d3. 

Maximum  flow 

G2d4. 

Test  problem  generation 

G2e. 

Quadratic  programming 

G2el. 

Positive  definite  Hessian  (i.e.,  convex  problem) 

G2e2. 

Indefinite  Hessian 

G2f. 

Geometric  programming 

G2g. 

Dynamic  programming 

G2h. 

General  nonlinear  programming 

G2hl. 

Simple  bounds 

G2h2. 

Linear  equality  or  inequality  constraints 

G2h3. 

Nonlinear  constraints 

G2i. 

Global  solution  to  nonconvex  problems 

G3. 

Optimal  control 

G4. 

Service  routines 

G4a. 

Problem  input  (e.g.,  matrix  generation) 

G4b. 

Problem  scaling 

G4c. 

Check  user-supplied  derivatives 

G4d. 

Find  feasible  point 

G4e. 

Check  for  redundancy 

G4f. 

Other 

H. 

Differentiation,  integration 

I. 

Differential  and  integral  equations 

J. 

Integral  transforms 

K. 

Approximation 

L. 

Statistics,  probability 
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M.  Simulation,  stochastic  modeling 

N.  Data  handling 

O.  Symbolic  computation 

P.  Computational  geometry 

Q.  Graphics 

R.  Service  routines 

S.  Software  development  tools 

Z.  Other 


Table  3.1  GAMS  Classification  System  at  Highest  Level  and  all  of  Optimization  Taxon 

A.  PROBLEM  AREA 

This  taxonomy  lists  the  specific  task  on  which  the  decision  maker  would  like 
support  from  a  DSS.  This  list  is  created  primarily  from  the  OR/MS  Index,  Encyclopedia 
of  OR/MS  ,  and  from  examples  of  DSS’s  online  or  developed  by  students  for  small  scale 
implementation.  One  of  the  most  difficult  classification  decisions  was  how  to  list  the  OR 
problems  that  have  nicknames  (knapsack  problem,  bin  packing)  which  have  one  distinct 
solution  method.  These  were  represented  under  the  optimization  hierarchy  in  the  solution 
methods  and  also  listed  under  problem  area  where  applicable.  The  final  taxonomy  of 
Problem  Area  or  specific  task  is  shown  in  Table  3.2. 

A.  Asset  pricing 

B.  Assignment 

C.  Bin  packing 

D.  Capital  budgeting 

E.  Communications  networks 

F.  Corporate  strategy 

G.  Costs  analysis 

H.  Crew  scheduling 

I .  Depreciation 

J.  Environment/ Agriculture  systems  analysis 

K.  Facilities/equipment  planning 

L.  Fire  models 

M.  Hierarchical  production  planning 

N.  Inventory 

O.  Investment 
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P.  Knapsack  problem 

Q.  Job  shop  scheduling 

R.  Layout/Location  of  equipment 

S .  Leaming/T  raining 

T.  Lifecycle 

U.  Location  analysis 

V.  Maintenance/Repair 

W.  Manpower  planning 

X.  Material  handling 

Y.  Planning 

AA.  Production 

AB.  Quality  control 

AC.  Queuing 

AD.  Reliability  of  systems 

AE.  Risk  assessment/management 

AF.  Safety 

AG.  Scheduling/sequencing 

AH.  Taxation 

AI.  Traffic  analysis 

AJ.  Utility/Preferences 

AK.  Vehicle  routing 

AL.  Yield  management 


Table  3.2  Problem  Area  Taxonomy 
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B.  FUNCTIONAL  AREA 

Functional  area  is  defined  as  the  component  parts  of  an  organization.  The 
characteristic  functional  area’  was  created  to  bridge  the  gap  between  industry  and 
problem  area,  since  tasks  are  not  elements  of  Industries  as  much  as  they  are  elements  of 
departments  or  functional  areas  within  Industries.  Also  this  provides  a  way  for  users  to 
search  objects  primarily  by  their  industry  and  functional  area  and  to  a  lesser  degree  by 
their  specific  problem  area,  to  see  what  is  available.  Conversely,  the  user  could  search  by 
problem  area  and  functional  area  ignoring  industry.  The  functional  area  listing  comes 
from  departments  of  industry,  government  and  military  organizations  as  suggested  by 
Yahoo  and  the  OR/MS  Index.  The  functional  area  listing  is  shown  in  Table  3.3. 


A. 

Administration 

B. 

Engineering 

C. 

Finance 

D. 

Legal 

E. 

Logistics 

F. 

Maintenance 

G. 

Marketing 

H. 

Payroll 

I. 

Personnel 

J. 

Public  Relations 

K. 

Sales 

L. 

Software  Delivery 

M. 

Supply 

N. 

Testing 

0. 

Training 

P. 

Transportation 

Q. 

T  elecommunications 

Table  3.3  Functional  Area  Taxonomy 
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C.  INDUSTRY 

The  industry  listing  defines  businesses  and  organizations  by  the  type  of  work  they 
do  or  branch  of  a  field  (such  as  manufacturing).  The  references  in  this  list  are  adapted 
from  the  Census  Bureau,  Yahoo,  the  OR/MS  Index,  OR/MS  Encyclopedia,  Reifer’s 
Application  Taxonomy  (Reifer  Consultants,  1990),  Digital  Corporation  (Digital  Corp., 
1991)  and  IBM’s  Industry  Taxonomies  (IBM  Corp.,  1988). 


A. 

Aerospace/Space 

B. 

Accounting 

C. 

Agriculture 

Cl. 

Crops 

C2. 

Food  Production 

C3. 

Livestock 

C4. 

Weather 

D. 

Arts 

Dl. 

Fine  Arts 

D2. 

Music 

D3. 

Theater 

E. 

Business/Commerce 

El. 

Apparel 

E2  . 

General  Retailing 

E3. 

Mailorder 

E4 . 

Restaurant 

E5. 

Wholesale 

F. 

Communications 

G. 

Construction 

Gl. 

Architecture 

G2. 

Materials 

H. 

Economics 

HI. 

Macroeconomics 

H2. 

Microeconomics 

I. 

Education 

11. 

K-12 

12. 

Undergraduate 

13. 

Graduate 

14. 

Career  training 

15. 

Vocational 

J. 

Emergency  Services 

Jl. 

Fire 

J2. 

Police 
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J3. 

Hospitals 

K. 

Energy 

L. 

Engineering 

M. 

Entertainment/Media 

Ml. 

Broadcasting 

M2. 

Film 

M3. 

Music 

M4. 

Publishing 

M5. 

Radio 

M6. 

Television 

N. 

Environment  and  Ecology 

Nl. 

Air  quality 

N2. 

Forestry 

N3. 

Pollution 

N4. 

Recycling 

N5. 

Pollution 

N6. 

Water  quality 

0. 

Finance 

01. 

Banking 

02. 

Insurance 

03. 

Investment 

04. 

Mortgage 

05. 

Personal 

06. 

Real  Estate 

P. 

Government 

PI. 

Elections 

P2. 

Energy  Policy 

P3. 

Public  Works 

P3. 

Regulations 

P4. 

Tax  Policy 

P5. 

Urban  Systems 

Q 

Health  /Medicine 

Ql. 

Exercise 

Q2. 

Health  care 

Q3. 

Nutrition 

R. 

International  Trade 

S. 

Labor 

T. 

Law 

U. 

Libraries 

Ul. 

Archives 

V. 

Manufacturing 

VI. 

Aerospace 
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V2.  Automotive 

V3.  Building  materials 

V4.  Chemical 

V5.  Clothing/Textiles 

V6.  Forest/  Paper  products 

V7.  Furniture 

V8.  Metals 

V9.  Pharmaceuticals 

V10.  Plastics 

W.  Marketing 

X.  Military 

XI .  Avionics 

X2.  Command  and  Control 

X3.  Data  processing 

X4.  Simulation 

X5.  Software  tools 

X6.  Telecommunications 

X7.  Testing 

Y.  Natural  Resources 

Z.  Petro-Chemical 

AA.  Population 

AB.  Recreation  and  Sports 

AC.  Scientific  Research 
AC1.  Chemistry 
AC2.  Geography 
AC3.  Engineering 
AC4.  Mathematics 
AC5.  Meteorology 
AC6.  Oceanography 
AC7.  Physics 
AC8.  Statistics 

AD.  Transportation 
ADI.  Airline 
AD2.  Automobile 
AD3.  Cargo 
AD4.  Railroad 
AD5.  Shipping 


Table  3.4  Industry  Taxonomy 
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D.  ORGANIZATION  TYPE 

This  characteristic  defines  the  kind  of  organization  for  which  the  object  was 
originally  designed.  An  organization  is  defined  as  an  association  of  people  brought 
together  for  a  particular  purpose.  The  organization  determines  how  objectives,  risk  and 
utility  are  weighed;  these  are  key  elements  in  decision  support  systems  and  decision 
analysis.  A  government  or  military  DSS  will  not  express  itself  in  terms  of  profitability  but 
by  service  or  “readiness”  objective  functions.  There  may  be  overlaps  between  the  different 
organization  types  and  in  many  instances  this  will  not  be  an  important  element  but  to  other 
objects  it  may  be  vital  to  the  correct  use  of  an  object.  Table  3.5  lists  the  organization 
types.  There  is  no  further  taxonomy  developed  for  this  characteristic,  but  specialization 
could  be  created  as  needed. 

A.  Military 

B.  Government  (non-military) 

C.  Commercial 

D.  Non-commercial/non-profit 

E.  Personal/Individual 

Table  3.5  Organization  Type  Taxonomy 
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E.  SOLUTION  METHOD 

The  solution  method  taxonomy  is  primarily  a  consolidation  of  parts  of  the  GAMS 
software  repository  classification  system  into  six  areas  with  the  inclusion  of  the 
knowledge  based,  spreadsheet  modeling  and  decision  analysis  techniques.  Table  3.6 
shows  the  top  level  of  the  solution  method  taxonomy.  The  full  taxonomy  to  four  levels 
deep  is  available  in  the  Appendix. 


A.  Simulation/  stochastic  modeling 

B.  Statistics/probability  -  problem  has  elements  of  uncertainty 

C.  Gaming 

D.  Optimization 

E.  Numerical  Math  -(non  optimization)/(non  statistics) 

F  .  Spreadsheet  modeling  (does  not  include  imbedded  optimization) 

formula/expression  evaluation 

G.  Knowledge  based 

H.  Symbolic  math  (calculus) 

I .  Decision  Analysis 

J.  Other 


Table  3.6  Solution  Method  Taxonomy 
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IV.  IMPLEMENTATION  OF  THE  TAXONOMY 


To  implement  the  taxonomy  classification  scheme  and  make  it  available  to  the 
users  it  had  to  become  part  of  the  system’s  user  interface.  The  user  interface  was  built 
using  hypertext  markup  language  (HTML)  and  common  gateway  interface  (CGI),  a  data 
handling  technique,  with  Netscape  2.0  as  the  de  facto  browser  standard.  The  available 
HTML  tools  and  those  being  developed  provide  the  primary  structure  for  building  the 
User  Interface.  Primary  HTML  tools  used  were  forms,  drop  down  menus  (vital  to 
browsing  and  selecting  taxonomies,)  and  “frames.”  In  fact,  the  proliferation  of  new 
browsers  and  capabilities  by  different  companies  posed  another  design  question.  For 
which  browsers  should  the  system  be  built?  -low  end  (lowest  common  denominator-  most 
users)  or  high  end  (greatest  capabilities,  more  tools-  but  more  maintenance  involved  in 
keeping  the  User  Interface  current  and  possibly  leaving  out  users  who  don’t  keep  then- 
browser  up  to  date.)  The  decision  was  to  build  for  the  high  end  and  greatest  capabilities. 
This  is  because  the  implementation  of  a  sophisticated  database,  software  agents  and  CGI 
functions  require  the  tools  available  at  the  higher  end  of  the  spectrum  of  Internet 
browsers.  Also  we  anticipate  our  primary  users  will  use  the  higher  end  Internet  browsers. 

A.  THE  DECISION  TO  ORGANIZE  DECISIONNET  AS  A  DATABASE 

SYSTEM 

The  DecisionNet  system  was  first  built  on  a  Unix  system  using  Perl  (Wall  and 
Schwartz,  1991)  scripts.  The  database  was  built  using  flat  file  searches  implemented  with 
a  Harvest  indexing  system  (Bowman  et  al.,  1994).  The  fundamental  difficulty  was  the 
inability  to  do  relevancy  searches  that  could  access  a  taxonomy  structure  using  that 
operating  system. 

In  analyzing  the  relationships  between  the  users,  providers,  and  objects,  and  the 
need  to  perform  relevancy  based  searches  we  determined  the  storage  of  the  object 
information  would  be  most  appropriately  handled  by  a  relational  database.  Delphi,  a  rapid 
application  development  (RAD)  tool,  with  a  Paradox  based  relational  database  is  used. 
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Delphi  uses  reusable  Pascal  scripts  and  many  CGI  components  that  can  direct  data 
transferring  from  remote  client  user  to  server.  (Calvert,  1995)  CGI  components  are 
quickly  being  developed  by  the  Internet  users  community. 

Based  on  the  equipment  and  infrastructure  that  is  currently  available  we  are: 

1.  creating  a  web  based  interface  (HTML  pages)  using  dimensions  and 
taxonomies  that  users  will  have  to  choose  from  for  their  search  and  registration  of 
objects. 

2.  linking  the  user  interface  to  the  server  applications  (administrative  functions 
and  using  registered  technologies,  probably  using  Delphi  and  SQL  queries). 

3.  developing  and  activating  search  mechanisms  (relevance  and  retrieval 
algorithms)  that  can  return  relevant  scores  on  all  dimensions  of  objects. 

The  entire  DecisionNet  database  consists  of  several  tables  to  do  administrative 
tasks  and  handle  security  functions.  The  basic  tables  consist  of: 

•  user  information 

•  provider  information 

•  object  information 

•  six  master  tables 

Tables  4. 1  and  4.2  illustrate  how  the  DecisionNet  system  searches  the  database 
fields  by  counting  the  distance  between  nodes  to  a  common  parent. 
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Name 

Parent 

Industry 

(None) 

Government 

Industry 

Manufacturing 

Industry 

Public  Works 

Government 

Tax  Policy 

Government 

Automotive 

Manufacturing 

Building  Materials 

Manufacturing 

Plastics 

BBH1IB 

Table  4. 1  Industry  Master  Table 


Name 

Parent 

Solution  Method 

(None) 

Knowledge  Based 

Solution  Method 

Numerical  Math 

Solution  Method 

Optimization 

Solution  Method 

Decision  Analysis 

Solution  Method 

Decision  Trees 

Decision  Analysis 

Unconstrained 

Optimization 

Constrained 

Optimization 

Constrained 

Table  4.2  Solution  Method  Table 


Each  “master”  table  lists  the  taxonomy  level  and  its  parent  level.  It  will  be  used  to 
generate  the  hierarchy  from  which  users  select  object  classification  information.  When 
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the  taxonomy  structure  changes  or  is  added  to  the  master  table  is  updated  once.  Rather 
than  changing  the  taxonomy  structure  on  several  HTML  pages  every  time  there  is  an 
update.  Ultimately,  HTML  pages  will  not  be  static  but  generated  on  the  fly  using  fields 
extracted  from  the  master  tables  as  required.  Field  names  with  the  same  parent  are 
siblings,  or  at  the  same  level  in  the  taxonomy. 

Forms  and  browsable  examples  of  the  hierarchy  are  generated  when  the  user 
selects  a  particular  characteristic  by  searching  the  master  table.  For  example,  when  the 
provider  or  consumer  selects  “Functional  Area”  all  those  fields  where  parent  is 
“Functional  Area”  are  displayed  in  a  browsable  manner  (like  the  frames  feature  in 
Netscape  shown  in  Figs.  4.1, 4.2  and  4.3).  If  the  user  then  selects  “Legal”  from  the  list  of 
children  of  “Functional  Area”  another  alphabetical  list  is  created  of  all  of  the  children  of 
“Legal”  and  this  goes  on  as  deep  as  the  particular  branch  goes  (from  one  to  six  levels 
deep).  The  consumer  defines  what  he  is  looking  for  by  selecting  the  characteristics  in  a 
similar  manner. 

The  mechanism  by  which  objects  are  retrieved  from  the  repository  involves 
searching  the  repository  for  an  exact  match  of  all  fields  of  the  user’s  search  parameters.  If 
all  fields  are  not  exactly  matched  then  the  system  scores  those  available  by  relevancy, 
showing  only  those  that  are  highly  relevant.  Relevancy  is  determined  by  calculating  the 
distance  between  the  object  the  user  seeks  and  objects  that  are  registered  in  the  repository. 
Those  elements  that  belong  to  the  same  parent  are  scored  the  highest  and  those  that  share 
a  parent  in  common  but  two  levels  higher  receive  a  lesser  relevant  score.  This  is,  of 
course,  going  on  in  each  of  the  six  characteristics.  These  concepts  were  discussed  in 
Chapter  II . 

To  make  the  database  available  with  up  to  the  minute  accuracy  and  high  relevancy 
we  use  batch  processing  of  queries  and  search  using  a  classification  scheme. 

B.  USER  INTERFACE 

Once  the  taxonomies  were  determined  and  the  system  foundation  built,  the  next 
concern  was  creating  the  User  Interface  to  let  the  user  know  what  the  system  can  do  and 
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how  to  access  those  capabilities. 

1.  Literature  Review 

A  literature  review  of  this  area  revealed  many  principles  on  which  a  system’s  user 
interface  should  be  developed.  Some  of  those  that  were  useful  included:  Donald 
Norman’s  book  User  Centered  System  Design:  New  Perspectives  on  Human-Computer 
Interaction  (1986),  Ramsey  and  Atwood’s  Human  Factors  in  Computer  Systems:  A 
Review  of  the  Literature  (1979),  Andriole’s  book  Handbook  of  Decision  Support  Systems 
(1989),  and  Carey’s  Human  Factors  in  Management  Information  Systems  (1988). 

2.  Principles  of  User  Interface  Design 

Some  of  the  guiding  principles  of  User  Interface  taken  from  these  books  are: 

•  Begin  by  profiling  the  intended  user 

•  Allow  user  to  enter  data  by  selection  not  entry 

•  Predictable  behavior  to  guide  the  new  user 

•  Use  as  few  keystrokes  or  steps  as  possible  to  perform  function 

(Carey,  1988) 

To  begin  with  the  first  principle  of  characterizing  the  user  and  his  skills,  Ramsey 
and  Atwood  identified  three  types  and  associated  characteristics: 

•  Naive  user 

•  Managerial  user 

•  Scientific-Technical  user 

Some  more  helpful  questions  for  identifying  the  user  and  their  use  of  the  system 
that  should  influence  system  design  are: 

“How  frequently  will  users  use  the  system? 

Will  the  system  be  used  by  users  under  situational  pressure? 

How  experienced  with  interactive  computing  are  the  users? 

How  experienced  with  analytical  methodology  are  the  users;  are  they  inclined  to 
think  analytically  or  are  they  more  passive  users?”(Andriole,  1989) 


33 


3.  Who  is  the  User? 

The  range  of  users  anticipated  are  managers,  operations  research  practitioners, 
students  of  OR,  and  students  of  decision  support  and  expert  systems.  Moreover,  we  want 
the  user  interface  to  be  appropriate  for  a  user  from  DoD  or  from  the  general  public 
seeking  a  decision  technology.  For  this  reason  we  also  want  the  search  procedures  to  be 
straightforward  enough  for  the  typical  Internet  user.  Users  of  DecisionNet  will  probably 
be  primarily  managerial  users.  Managerial  users  are  characterized  as  having:  variable 
information  needs,  very  low  tolerance  for  system  “impedance”.  If  dissatisfied  with  the 
user  interface  they  will  use  it  less  or  not  at  all  or  have  someone  else  use  it  to  perform  the 
functions  they  need.  (Carey,  1988)  The  point  about  variable  information  needs  is  a 
reminder  that  these  managers  will  probably  be  well  educated  but  from  diverse 
backgrounds  and  have  vocabulary  and  definitions  of  key  terms  based  on  their  field  or 
background. 

There  may  also  be  naive  users  of  the  system  who  need  an  online  tutorial  (which 
can  be  provided  through  online  documentation).  Scientific-technical  users  will  have  very 
proprietary  definitions  of  key  terms  and  would  benefit  from  documentation  explaining  the 
use  of  terms  and  the  design  of  system.  In  fact,  any  application  should  have  at  a  minimum 
strong  feedback  mechanisms  and  built  in  documentation,  regardless  of  the  users’  skill 
level. 

We  are  designing  the  system  based  on  the  following  assumptions  about  the  users 
and  their  skill  levels: 

Internet  user  knows  how  to  point  and  click  and  is  comfortable  with  Netscape  user 
interface,  drop  down  select  boxes  or  frames.  The  user  may  not  know  anything  about 
decision  support  systems  except  what  has  been  explained  on  entry  pages. 

Undergraduate/Graduate  student-  Information  Systems  or  Operations  Research 
student  who  is  looking  for  DSS’s.  This  user  may  need  clarification  on  terms  and  solution 
method  taxonomy. 

Manager  (military  or  civilian)  This  user  is  looking  for  a  DSS  or  expert  system  for  a 
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specific  application  in  their  industry  or  problem  area.  Solution  method  probably  isn’t 
important  but  as  mentioned  earlier  this  user  will  have  difficulty  with  distinctions  made  in 
the  vocabulary. 

Academia-  Familiar  with  Internet  and  decision  support  systems  but  may  need 
clarification  of  terms  and  definitions.  This  user  is  likely  to  be  frustrated  if  user  interface 
restricts  what  they  want  to  do  and  would  appreciate  the  capability  to  jump  between  areas 
they  are  browsing. 

The  minimum  skill  set  is  relatively  high  because  DSS’s  are  management  tools  and 
this  is  an  educated  user  group.  Users  may  not  have  a  sufficient  background  in  OR  to 
select  a  solution  method  from  a  complex  classification  of  systems.  Solution  method 
classification  will  be  of  primary  interest  to  academia  and  is  essential  to  provider 
registration  of  objects. 

4.  Application  of  User  Interface  Design  Principles 

The  principles  listed  previously  were  specifically  applied  to  DecisionNet  through 
the  User  Interface  design.  Users  select  rather  than  create  items  for  data  entry  from 
dropdown  menus  for  classification  of  objects.  DecisionNet  pages  and  methods  for 
executing  functions  provide  for  a  consistent  user  interface  and  meaningful  feedback. 

These  qualities  help  the  user  to  learn  system  and  how  it  works.  The  use  of  Netscape’s 
Internet  browser,  forms  and  dropdown  menus  minimizes  the  number  of  steps  the  user  has 
to  perform.  Most  are  initiated  by  pointing  and  clicking  by  the  user.  Forms  provide  the 
perfect  vehicle  for  capturing  information  because  they  are  a  familiar  user  interface  (people 
know  what  is  asked  of  them  and  how  to  reply)  and  they  are  a  logical  structure  for  handling 
database  field  names.  Also  the  dropdown  menus  used  with  forms  minimize  the  amount  of 
information  users  have  to  remember  between  browsing  the  taxonomy  and  stating  their 
search  parameters. 

The  user  interface  was  implemented  with  some  of  the  tools  and  capabilities  of 
HTML  and  Netscape  Internet  browsers. 
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C.  PRESENTING  TAXONOMIES  AND  CLASSIFICATIONS  TO  USERS 

A  challenge  for  this  element  of  the  thesis  is  to  effectively  represent  the  taxonomy 
to  the  users.  During  this  thesis  work,  more  and  more  tools  became  available  for  the 
implementation  of  a  navigatable  tree  using  Netscape  browser  software.  The  first  tool  was 
“frames”  which  became  widely  available  when  Netscape  2.0  was  in  testing  and  released  in 
February  1996.  The  taxonomy  representation  structure  stored  every  taxon  by  its  own 
name  and  that  of  its  parent,  in  a  table  as  shown  in  Tables  4. 1  and  4.2.  When  “Industry”  is 
selected,  all  of  the  elements  in  the  Industry  master  table  with  “Industry”  as  a  parent  are 
displayed.  When  “Manufacturing”  is  selected,  all  of  the  elements  in  the  Industry  master 
table  with  “Manufacturing”  as  the  parent  are  shown.  This  was  adapted  to  our  relational 
database  design  and  the  taxonomy  representation  which  can  be  seen  in  Fig.  4. 1,  4.2  and 
4.3. 
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Figure  4.1  All  Taxonomies  at  Level  0 
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Figure  4.2  Industry  Taxonomy  at  Level  1 
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Because  the  DecisionNet  user  is  remote  and  navigating  the  taxonomy  to  create 
their  search  parameters,  a  difficult  aspect  of  this  implementation  was  keeping  memory  of 
past  selections  between  the  online  forms  and  taxonomy  representation.  The  database 
structure  (shown  in  Fig  4.1,  4.2  and  4.3)  is  fine  for  browsing  the  taxonomy,  but  becomes 
complicated  when  a  selection  made  while  browsing  results  in  the  execution  of  several 
scripts  and  database  queries  on  the  DecisionNet  system.  This  is  because  the  transfer  of 
files  from  server  to  client  using  the  World  Wide  Web  is  a  onetime  transaction;  the  packet 
is  sent  and  the  brief  connection  broken.  Each  new  request  generates  a  new  attempt  to  log 
into  the  server  and  establishes  a  new  connection.  Maintaining  state  between  views  of  the 
taxonomy  is  an  important  part  of  the  user  interface  because  of  the  complexity  of  sorting 
through  six  taxonomies.  Because  of  the  complexity  of  passing  elements  of  data  through 
CGI  and  the  users  unstructured  browse  technique,  the  DecisionNet  development  team  is 
waiting  for  the  arrival  of  another  Delphi  companion  product  called  “WebHub”  developed 
specifically  for  this  type  of  problem.  This  problem  is  called  “maintaining  state”  while  a 
user  is  logged  into  and  browses  a  site  generating  multiple  queries  and  scripts  to  execute. 
The  developer  Ann  Lynnworth  compares  it  to  “keeping  track  of  the  items  a  shopper  puts 
in  their  grocery  cart”,  (Lynnworth,  1995)  which  is  something  current  “onetime  submit” 
forms  do  not  do  well. 

D.  GETTING  INFORMATION  FROM  THE  USER 

The  User  Interface  to  DecisionNet  is  a  series  of  HTML  pages  that  the  user 
navigates  beginning  with  their  login  to  DecisionNet.  The  initial  HTML  pages  are  user  or 
provider  registration  and  user  or  provider  login.  These  are  static  HTML  pages  using 
forms  and  table  HTML  tools.  The  Delphi  script  executes  and  either  accepts  or  rejects  the 
login  or  registration  information.  Other  functions  are  offered  to  users  after  the  login  is 
verified  against  previously  entered  login  information.  Ideally  these  pages  providing  other 
functions  would  not  actually  exist  on  the  server  but  would  be  created  for  users  to  access 
when  a  user  is  correctly  logged  in  (to  ensure  that  security  or  billing  functions  are  being 
enforced).  This  is  also  because  the  bulk  of  the  information  they  contain  is  generated  from 
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the  current  database  structure.  For  this  implementation  phase  and  to  get  the  scripts 
working  I  created  actual  .html  files  on  the  server  that  the  Delphi  script  points  to  when  it 
has  successfully  executed.  This  is  done  by  passing  HTML  commands  to  the  server 
browser,  which  it  displays.  Because  we  want  the  user  to  select  from  a  list  of  choices  for 
each  classifying  characteristic,  drop  down  lists  on  registration  and  search  forms  are  used. 
This  eliminates  the  opportunity  to  misspell  or  forget  which  choice  is  appropriate  while 
switching  from  one  HTML  page  to  another.  An  example  of  a  dropdown  list  on  a  form  is 
shown  in  Fig.  4.4. 


Figure  4.4  Example  of  Dropdown  Menu  on  the 
Registration  Page 


The  ideal  (though  complicated  to  code)  interface  would  be  navigatable  frames 
within  a  form  that  build  the  taxonomies  for  a  search  and  perform  possibly  several  searches 
in  a  row  while  staying  at  what  is  essentially  the  same  first  page.  Using  frames  tools  a 
static  shell  in  the  margins  could  be  much  like  the  toolbars  and  powerbars  in  Windows- 
based  applications.  In  this  example,  the  content  changes  but  the  framework  stays  the 
same,  which  is  pleasing  as  a  user  interface  and  provides  a  consistent  location  for  functions, 
definitions  and  online  help. 
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Frames  (as  shown  in  Fig.  4.1,  4.2  and  4.3)  can  be  used  to  implement  online 
documentation  which  will  be  helpful  to  users  in  deciding  what  they’re  looking  for.  A 
distinct  advantage  of  using  frames  is  it  will  allow  tutorials  to  not  be  disruptive  to 
advanced  users.  Thoroughness  to  inexperienced  users  can  be  distracting  to  experienced 
users.  Moreover,  the  use  of  frames  and  HTML  pages  gives  online  help  for  other 
questions,  further  explanation,  and  in  the  future,  online  tutorial  or  help  screens  through 
frames.  Another  possibility  for  DecisionNet  is  the  opportunity  to  send  the  entire 
taxonomy  classification  to  the  client  as  an  executable  program  within  the  browser.  The 
user  could  then  navigate  the  taxonomy  and  make  their  selections  and  send  back  their 
search  parameters  to  the  DecisionNet  system.  In  this  scenario  the  user  queries  the 
DecisionNet  server  and  database  once,  rather  than  reconstruct  the  taxonomy  classification 
and  query  the  database  to  navigate  the  taxonomy  several  times  per  user  session. 

HTML  tools  are  available  for  building  an  online  system  that  is  much  more  than  a 
terminal  linking  to  a  distant  computer  but  a  system  that  provides  a  clear,  meaningful  and 
instructive  user  interface  to  decision  support  tools  and  helps  users  find  objects  they  may 
not  have  realized  were  available. 
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V.  CONCLUSIONS 


A.  SUMMARY 

The  DecisionNet  system  aims  to  provide  decision  makers  and  researchers  global 
access,  over  the  Internet,  to  a  large  distributed  collection  of  decision  technologies.  Recent 
research  in  OR/MS  and  Information  Technology  involves  several  other  efforts  with 
similar  objectives.  Successful  use  of  such  repositories  depends,  to  a  large  extent,  on  the 
users  being  able  to  search  this  large  collection  to  locate  relevant  technologies.  This 
depends,  in  turn,  on  i)  a  suitable  indexing  and  classification  scheme  along  which  the 
technologies  are  organized,  and  ii)  intelligent  search  and  retrieval  algorithms  for 
computing  relevance  of  technologies  against  given  search  criteria. 

This  thesis  has  addressed  the  first  part  of  this  problem.  Specifically  we  have 
developed  and  constructed  taxonomies  to  allow  for  indexing  and  classification  of  decision 
technologies  in  the  repository.  These  technologies  include  data  sets,  algorithms,  models 
and  decision  support  systems.  Previous  research  in  this  area  has  resulted  in  the 
development  of  few  indices  of  decision  technologies;  the  most  notable  effort  is  the  Guide 
to  Available  Mathematical  Software  (Boisvert,  1991).  Another,  Yahoo  (Yahoo,  1995),  is 
a  thorough  subject  indexing  scheme.  However,  these  index  hierarchies  are  single 
dimensional,  in  most  cases  the  dimension  is  the  solution  methodology  used  in  the 
technology. 

An  early  conclusion  of  our  research  was  that,  in  order  to  be  effective  from  a  user's 
point  of  view,  technologies  had  to  be  classified  along  several  different  dimensions.  Five 
primary  dimensions  were  identified:  these  are  functional  area,  problem  area,  industry, 
organization  type  and  solution  method.  These  dimensions  represent  the  likely  starting 
points  of  a  user's  search  for  a  suitable  technology  and  encompass  search  requirements  for 
a  diverse  group  of  users  including  researchers,  analysts,  and  end  users.  The  detailed 
taxonomies  were  developed  by  combining  the  terms  in  existing  taxonomies  with  criteria 
and  principles  for  taxonomy  development.  A  database  architecture  was  developed  to  store 
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both  the  taxonomies  (including  structural  relationships  between  terms  in  the  taxonomies) 
and  the  decision  technology  objects  in  the  repository.  A  WWW-capable  user  interface  was 
also  developed  to  allow  providers  to  register  and  index  their  technologies  and  to  allow 
users  to  browse  through  the  taxonomies  or  to  specify  search  criteria. 

B.  FURTHER  RESEARCH 

As  mentioned  above,  this  thesis  addressed  one  part  (development  of  a 
classification  scheme)  of  the  indexing  and  retrieval  problem  for  a  repository  of  decision 
technologies.  Further  research  is  needed  on  the  second  part,  to  develop  intelligent  search 
and  retrieval  algorithms.  The  basic  tradeoff  in  this  area  is  between  recall  and  relevance.  An 
algorithm  that  insists  on  a  very  tight  match  will  probably  find  highly  relevant  objects,  but  it 
may  fail  to  recall  others  that  are  relevant.  An  algorithm  that  applies  several  relaxation  rules 
(e.g.,  synonyms,  generalizations)  in  conducting  the  search  will  recall  many  more  objects, 
but  many  of  them  may  not  be  relevant.  The  ultimate  test  of  a  classification  scheme  is  how 
useful  it  is  in  solving  the  user's  search/retrieval  problem.  Several  algorithms  need  to  be 
developed  and  tested  in  order  to  determine  the  usefulness  of  the  classification  scheme  laid 
out  in  this  thesis. 

Another  area  of  research  involves  the  development  of  software  agents  to  perform 
administrative  and  maintenance  tasks  on  the  classification  scheme  itself.  Due  to  the  nature 
of  decision  technologies  any  classification  scheme  will  need  to  undergo  changes  over  time. 
How  will  such  changes  affect  the  existing  structures  and  the  classifications  already  made 
with  the  existing  taxonomies?  With  good  maintenance  agents,  these  changes  should  be 
achieved  with  minimal  human  intervention. 

This  thesis  has  assumed  a  user  interface  based  on  the  World  Wide  Web’s  client- 
server  model.  For  example,  a  provider  wishing  to  index  a  technology  at  a  lower  level  in 
the  taxonomy  may  have  to  send  several  messages  to  the  index  server.  It  may  be  useful  to 
investigate  other  mechanisms  for  accomplishing  such  tasks.  For  example,  the  Java 
language  may  be  used  to  create  an  applet  that  is  transferred  to  the  provider's  machine  and 
which  allows  the  provider  to  browse  through  the  taxonomy  levels  without  having  to  send 
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successive  requests  to  the  index  server. 

C.  CONCLUSIONS 

The  main  contribution  of  this  thesis  is  in  creating  taxonomies  that  can  be  used  to 
classify  decision  support  systems.  This  taxonomy  development  has  involved  the 
consideration  of  the  viewpoints  of  several  different  kinds  of  users,  issues  in  user  interface 
design,  as  well  as  a  systematic  application  of  principles  of  taxonomy  development.  While 
this  work  has  been  done  in  the  context  of  decision  technologies,  some  of  the  ideas 
presented  here  may  be  generalized  to  other  kinds  of  information  and  physical  products 
offered  on  the  Internet. 

The  increase  in  the  amount  of  information  and  in  the  number  of 
information-publishers,  and  the  exponential  growth  of  Internet  users,  makes  the  indexing 
and  retrieval  problem  more  critical.  The  practice  of  Information  Brokering  that  charges 
users  for  accurate,  verified  and  relevant  information  and  for  search  tools  will  undoubtedly 
increase.  A  strong  and  robust  classification  system  is  vital  to  the  effort  to  organize  and 
search  information  about  a  collection  of  objects. 
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APPENDIX:  SOLUTION  METHOD  TAXONOMY 


A  Simulation/  stochastic  modeling 
Al.  Simulation 
Ala.  Discrete 

Alb.  Continuous  (Markov  models) 

A2.  Queueing 
A3.  Reliability 
A3a.  Quality  control 

A3b.  Electrical  network 

B  Statistics/probability  -  problem  has  elements  of  uncertainty 
B 1 .  Aproximation 

Bla.  Least  squares  (L_2)  approximation 

B 1  a  1 .  Linear  least  squares 

B 1  a2 .  Nonlinear  least  squares 

Bib.  Minimax  (L_infmity)  approximation 

Blc.  Least  absolute  value  (L_l)  approximation 

Bid.  Other  analytic  approximations  (e.g.,  Taylor  polynomial,  Pade) 

Ble.  Smoothing 

B 1  f.  Service  routines  for  approximation 

B 1  fl .  Evaluation  of  fitted  functions,  including  quadrature 
Big.  Grid  or  knot  generation 

Blh.  Manipulation  of  basis  functions  (e.g.,  evaluation,  change  of  basis) 

Bli.  Other 

B2.  Statistics,  Probability 

B2a.  Data  summarization 

B2a  1 .  One-dimensional  data 

B2a2.  Two  dimensional  data 

B2a3.  Multi-dimensional  data 

B2b.  Data  manipulation 

B2b  1 .  Transform  (search  also  classes  LI Oal ,  N6,  and  N8) 

B2b2.  Tally 

B2b3.  Subset 

B2b4.  Merge 

B2b5.  Construct  new  variables  (e.g.,  indicator  variables) 

B2c.  Elementary  statistical  graphics 

B2c  1 .  One-dimensional  data 

B2c2.  Two-dimensional  data 

B2c3.  Three-dimensional  data 

B2c4.  Multi-dimensional  data 

B2d.  Elementary  data  analysis 
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B2dl.  One-dimensional  data 

B2d2.  Two-dimensional  data 

B2d3.  Multi-dimensional  data 

B2d5.  Multiple  Multi-dimensional  data  sets 

B2e.  Function  evaluation 

B2el.  Univariate 

B2e2.  Multivariate 

B2f.  Random  number  generation 

B2fl .  Univariate 

B2f2.  Multivariate 

B2f3.  Service  routines  (e.g.,  seed) 

B2g.  Analysis  of  variance  (including  analysis  of  covariance) 

B2gl .  One-way 

B2g2.  Two-way 

B2g3.  Three-way  (e.g.,  Latin  squares) 

B2g4.  Multi-way 

B2g5.  Multivariate 

B2g6.  Generate  experimental  designs 

B2g7.  Service  routines 

B2h.  Regression 

B2hl .  Simple  linear  (i.e.,  y  =  b_0  +  b_lx) 

B2h2.  Polynomial  (e.g.,  y  =  b_0  +  b_lx  +  b_2  xA2) 

B2h3.  Multiple  linear  (i.e.,  y  =  b_0  +  b_l  x_l  +  ...  +  b_p  xjp) 
B2h4.  Polynomial  in  several  variables 

B2h5.  Nonlinear  (i.e.,  y  =  F(X,b)) 

B2h6.  Simultaneous  (i.e.,  Y  =  Xb) 

B2h7.  Spline  (i.e.,  piecewise  polynomial) 

B2h8.  EDA  (e.g.,  smoothing) 

B2h9.  Service  routines  (e.g.,  matrix  manipulation  for  variable 

selection) 

B2i.  Categorical  data  analysis 

B2il.  2-by-2  tables 

B2i2.  Two-way  tables 

B2i3.  Log-linear  model 

B2i4.  EDA  (e.g.,  median  polish) 

B2j.  Time  series  analysis 

B2j  1 .  Univariate 

B2j2.  Two  time  series 

B2k.  Correlation  analysis  (search  also  classes  L4  and  LI 3c) 

B21.  Discriminant  analysis 

B2m.  Covariance  structure  models 

B2n.  Cluster  analysis 
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B2nl.  One-way 

B2n2.  Two-way 

B2n3.  Display 

B2n4.  Service  routines  (e.g.,  compute  distance  matrix) 

B2o.  Life  testing,  survival  analysis 

B2p.  Multidimensional  scaling 

B2q.  Statistical  data  sets 

C  Gaming  -  models  adversarial  conflict  between  intelligent  opponents  /Game 
theory 

D  Optimization  and  Linear  Algebra 
Dl.  Linear  Algebra 

Dial .  Elementary  vector  and  matrix  operations 
D 1  a2 .  Elementary  matrix  operations 

Dlb.  Solution  of  systems  of  linear  equations  (including  inversion,  LU 
and  related  decompositions) 

D 1  b  1 .  Real  nonsymmetric  matrices 

Dlb2.  Real  symmetric  matrices 

D 1  b3 .  Complex  non-Hermitian  matrices 

Dlb4.  Complex  Hermitian  matrices 

Dlb5.  Associated  operations  (e.g.,  matrix  reorderings) 

Die.  Determinants 

D 1  c  1 .  Real  nonsymmetric  matrices 

Dlc2.  Real  symmetric  matrices 

D 1  c3 .  Complex  non-Hermitian  matrices 

D 1  c4 .  Complex  Hermitian  matrices 

Did.  Eigenvalues,  eigenvectors 

Dldl .  Ordinary  eigenvalue  problems  (Ax  =  lambda  x) 

Dld2.  Generalized  eigenvalue  problems  (e.g..  Ax  =  lambda  Bx) 

D 1  d3 .  Associated  operations 

Die.  QR  decomposition,  Gram-Schmidt  orthogonalization 

D 1  f.  Singular  value  decomposition 

Dig.  Update  matrix  decompositions 

Dlgl.  LU 

Dlg2.  Cholesky 

Dlg3.  QR 

D 1  g4 .  Singular  value 

Dlh.  Other  matrix  equations  (e.g.,  AX+XB=C) 

Dli.  Singular,  overdetermined  or  underdetermined  systems  of  linear 
equations,  generalized  inverses 
D 1  i  1 .  Unconstrained 

Dli2.  Constrained 

D 1  i3 .  Generalized  inverses 
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D2  Optimization 

D2a.  Unconstrained 

D2al .  Univariate 

D2a2.  Multivariate 

D2b.  Constrained 

D2b  1 .  Linear  programming 

D2b2.  Transportation  and  assignments  problem 

D2b3 .  Integer  programming 

D2b3a.  Zero/one 

D2b3b.  Covering  and  packing  problems 

D2b3c.  Knapsack  problems 

D2b3d.  Matching  problems 

D2b3e.  Routing,  scheduling,  location  problems 

D2b3f.  Pure  integer  programming 

D2b3g.  Mixed  integer  programming 

D2b4.  Network  (for  network  reliability  search  class  M) 

D2b4a.  Shortest  path 

D2b4b.  Minimum  spanning  tree 

D2b4c.  Maximum  flow 

D2b4d.  Test  problem  generation 

D2b5  Quadratic  programming 

D2b6.  Geometric  programming 

D2b7.  Dynamic  programming 

D2b8.  General  nonlinear  programming 

D2b9.  Global  solution  to  nonconvex  problems 

D2c.  Optimal  control 

D2d.  Service  routines 

D2dl .  Problem  input  (e  g.,  matrix  generation) 

D2d2.  Problem  scaling 

D2d3 .  Check  user-supplied  derivatives 

D2d4.  Find  feasible  point 

D2d5  Check  for  redundancy 

D2d6.  Other 

E  Numerical  Math  -(non  optimization)/(non  statistics) 

E 1 .  Arithmetic,  error  analysis 

El  a.  Integer 

Elb.  Rational 

Elc.  Real 

E 1  c  1 .  Standard  precision 

E 1  c4 .  Extended  precision 

E 1  c4 .  Extended  range 

Eld.  Complex 
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E 1  d  1 .  Standard  precision 

Eld3.  Extended  precision 

Eld4.  Extended  range 

Ele.  Interval 

Elf.  Change  of  representation 

E 1  fl .  Type  conversion 

E 1  £2 .  Base  conversion 

El £3.  Decomposition,  construction 

Elg.  Sequences  (e.g.,  convergence  acceleration) 

E2.  Number  theory 

E3.  Elementary  and  special  functions 

E3a.  Integer-valued  functions  (e.g.,  factorial,  binomial  coefficient, 

permutations,  combinations,  floor,  ceiling) 

E3b.  Powers,  roots,  reciprocals 

E3c.  Polynomials 

E3cl.  Orthogonal 

E3c2.  Non-orthogonal 

E3d.  Elementary  transcendental  functions 

E3  d  1 .  Trigonometric,  inverse  trigonometric 

E3d2.  Exponential,  logarithmic 

E3d3.  Hyperbolic,  inverse  hyperbolic 

E3d4.  Integrals  of  elementary  transcendental  functions 

E3e.  Exponential  and  logarithmic  integrals 

E3f.  Cosine  and  sine  integrals 

E3g.  Gamma 

E3gl .  Gamma,  log  gamma,  reciprocal  gamma 

E3g2  .  Beta,  log  beta 

E3g3.  Psi  function 

E3g4 .  Polygamma  function 

E3g5.  Incomplete  gamma 

E3g6.  Incomplete  beta 

E3g7.  Riemann  zeta 

E3h.  Error  functions 

E3hl .  Error  functions,  their  inverses,  integrals,  including  the  normal 
distribution  function 
E3h2.  Fresnel  integrals 

E3h3.  Dawson's  integral 

E3i.  Legendre  functions 
E3j.  Bessel  functions 

E3jl.  J,  Y,  H_l,  H_2 

E3j2.  I,  K 

E3j3.  Kelvin  functions 
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E3j4.  Airy  and  Scorer  functions 

E3j5.  Struve,  Anger,  and  Weber  functions 

E3j6.  Integrals  of  Bessel  functions 

E3k.  Confluent  hypergeometric  functions 

E31.  Coulomb  wave  functions 

E3m.  Jacobian  elliptic  functions,  theta  functions 

E3n.  Elliptic  integrals 

E3o.  Weierstrass  elliptic  functions 

E3p.  Parabolic  cylinder  functions 

E3q.  Mathieu  functions 

E3r.  Spheroidal  wave  functions 

E3s.  Other  special  functions 

E4.  Interpolation 

E4a.  Univariate  data  (curve  fitting) 

E4al .  Polynomial  splines  (piecewise  polynomials) 

E4a2.  Polynomials 

E4a3.  Other  functions  (e.g.,  rational,  trigonometric) 

E4b.  Multivariate  data  (surface  fitting) 

E4bl.  Gridded 

E4b2.  Scattered 

E4c.  Service  routines  for  interpolation 

E4cl .  Evaluation  of  fitted  functions,  including  quadrature 

E4c2.  Grid  or  knot  generation 

E4c3.  Manipulation  of  basis  functions  (e.g.,  evaluation,  change  of 

basis) 

E4c4.  Other 

E5 .  S  olution  of  nonlinear  equations 
E5a.  Single  equation 
E5al .  Polynomial 

E5a2.  Nonpolynomial 

E5b.  System  of  equations 

E5c.  Service  routines  (e.g.,  check  user-supplied  derivatives) 

E6.  Aproximation 

E6a.  Least  squares  (L_2)  approximation 

E6al .  Linear  least  squares 

E6a2.  Nonlinear  least  squares 

E6b.  Minimax  (L_infinity)  approximation 

E6c.  Least  absolute  value  (L_l)  approximation 

E6d.  Other  analytic  approximations  (e.g.,  Taylor  polynomial,  Pade) 

E6e.  Smoothing 

E6f.  Service  routines  for  approximation 

E6fl .  Evaluation  of  fitted  functions,  including  quadrature 
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E6g.  Grid  or  knot  generation 

E6h.  Manipulation  of  basis  functions  (e.g.,  evaluation,  change  of  basis) 

E6i.  Other 

E6al.  Unconstrained 

E6a2.  Constrained 

E6b.  Nonlinear  least  squares 

E6b  1 .  Unconstrained 

E6b2.  Constrained 

F  Spreadsheet  modeling  ( not  include  imbedded  optimization)  formula/expression 
evaluation 

G  Knowledge  based 
H  Symbolic  math  (calculus) 

HI .  Elementary  and  Special  Functions 

Hla.  Integer-valued  functions  (e.g.,  factorial,  binomial  coefficient, 

permutations,  combinations,  floor,  ceiling) 

Hlb.  Powers,  roots,  reciprocals 

Hlc.  Polynomials 

Hlcl.  Orthogonal 

Hlc2.  Non-orthogonal  available  only  on  grid 

H2.  Diffemetiation,  integration 

H2a. 

H2b.  Multidimensional  integrals 

H2bl .  One  or  more  hyper-rectangular  regions  (includes  iterated 
integrals) 

H2b2.  n-dimensional  quadrature  on  a  nonrectangular  region 
H2c.  Service  routines  (e.g.,  compute  weights  and  nodes  for  quadrature 
formulas) 

H3.  Differential  and  integral  equations 
H3a.  Ordinary  differential  equations  (ODE's) 

H3  a  1 .  Initial  value  problems 

H3a2.  Multipoint  boundary  value  problems 
H3a3.  Service  routines  (e.g.,  interpolation  of  solutions,  error 
handling,  test  programs) 

H3b.  Partial  differential  equations 

H3b  1 .  Initial  boundary  value  problems 

H3b2.  Elliptic  boundary  value  problems 

H3c.  Integral  equations 

H4.  Integral  transforms 

H4a.  Trigonometric  transforms  including  fast  Fourier  transforms 

H4a  1 .  One-dimensional 

H4a2.  Multidimensional 

H4b.  Convolutions 
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H4c.  Laplace  transforms 
H4d.  Hilbert  transforms 
I  Decision  Analysis 


11. 

Decision  Trees 

12. 

Utility  Theory 

13. 

Influence  Diagrams 

14. 

AHP 

15. 

Risk  Analysis 

J  Other 
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